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Abstract 

In this paper we give an explicit construction of a capacity achieving family of binary 
t-write WOM codes for any number of writes t, that have a polynomial time encoding 
and decoding algorithms. The block length of our construction is 
when e is the gap to capacity and encoding and decoding run in time N 1+s . This is 
^yy the first deterministic construction achieving these parameters. Our techniques also 

apply to larger alphabets. 

1 Introduction 

In |RS82j Rivest and Shamir introduced the notion of write- once-memory and showed its 
relevance to the problem of saving data on optical disks. A write-once-memory, over the 
binary alphabet, allows us to change the value of a memory cell (say from to 1) only once. 
Thus, if we wish to use the storage device for storing t messages in t rounds, then we need 
to come up with an encoding scheme that allows for t-write such that each memory cell 
t— i is written at most one time in the binary setting, or it is written monotonically increasing 

values in the g-ary setting. An encoding scheme satisfying these properties is called a t-write 
Write-Once-Memory code, or a t-write WOM code for short. This model has recently gained 
renewed attention due to similar problems that arise when using flash memory devices. We 
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refer the readers to [Y KS+Tol IGYD+llj for a more detailed introduction to WOM codes and 



their use in encoding schemes for flash memory. 

One interesting goal concerning WOM codes is to find codes that have good rate for 
t- write. Namely, to find encoding schemes that allow to save the maximal information- 
theoretic amount of data possible under the write-once restriction. Following [RS82] it was 
shown that the capacity (i.e. maximal rate) of t-write binary WOM code is[j]log(t + 1) (see 
[RS82J IHee85t IFV99] ). Stated differently, if we wish to use an n-bit memory t-times then 
each time we can store, on average, n ■ log(t + l)/t many bits. 
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In this work we give, for any number of writes t, an explicit construction of an encoding 
scheme that achieves the theoretical capacity. Before describing our results we give a formal 
definition of a t-write WOM code. 

For two vectors of the same length y and y' we say that y' < y if y^ < yi for every 
coordinate %. 

Definition 1.1. A t -write binary WOM of length n over the sets of messages fii, 1^2; • • • > 
consists oft encoding functions Ei,...,E t such that Ex : Qi — > {0, l} n , E2 '■ Ei(Q\) x 
Q 2 {0, l} n and in general, Ei : Image{Ei_\) x Q { — > {0, l} n , and t decoding functions 
Di : Image(Ei) — > that satisfy the following properties^ 

1. For all 1 <i <t, c G lmage[Ei-.\) and Xi G Qi, Di(Ei(c,Xi)) = X{. 

2. For all 1 <i <t, c G Image(Ei_i) and Xi G Qi we have that c < Ei(c,Xi). 

The rate of such a WOM code is defined to be (log |Oi| + log |f2 2 | + . . . + log \ Q t \)/n. 

Intuitively, the definition enables the encoder to use E\ as the encoding function in the 
first round. If the message x\ was encoded (as the string E\{x\j) and then we wished to 
encode in the second round the message X2, then we write the string E 2 (Ei(xi), X2), etc. 
Since Ei(x\) < E2(Ei(x\), X2), we only have to change a few zeros to ones (in the binary 
setting) in order to move from E\{x\) to ^2 (-^1(^1)) x 2)- Similarly for any one of the t writes. 
The requirement on the decoding functions guarantees that at each round we can correctly 
decode the memory]^] Notice, for example, that in the ith round we are only required to 
decode Xi and not any of the earlier messages. 

Similarly, one can also define WOM codes over other alphabets. 

The capacity region of a binary t-write binary WOM code was shown in |RS82l IFV99[ 
IHee85j to be the set 

C t = I (R u . . . , Rt) G R% I R x < H( Pl ), R 2 <(1- Pi)H(p 2 ), R 3 < - Pl )H(p 3 ), 
{ »=i 

t-2 t-1 

■ • • ' ^-1 ^ n^ 1 - PiWpt-i), ^ < n^ 1 - where ° < pu ■ ■ ■ < 1/2 



1=1 i=l 



From this it is not hard to show that the maximal rate of a i-write WOM code is log(£ + 1) 
and that this rate is achieved by the point 

(R U ...,R,)= (h (-!—) , (-\ , t f±H (-!—) -^-H ^ 



t + lj t + l KtJ't + l \t-lj 't+1 V37 t 



2 To simplify notation let us denote by Eq a map whose image is Image(i5o) = 0. We also abuse notation 
and denote Bi(0, x) = E\{x). 

3 We implicitly assume that the decoder knows, given a codeword, the round in which it was encoded. At 
worst this can add another t-bits to the encoding which has no real effect (in the asymptotic sense) on the 
rate. 
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in the capacity region (corresponding to the choice (pi, . . . ,Pt-i) = (^pr> | > • • • > |))- For 
a point (Ri,...,Rt) G C t we call the vector (pi, . . . G [0, the corresponding 

weight vector. Namely, (pi, . . . ,Pt-i) is a corresponding weight vector if Ri < H(pi), R2 < 
(1 - Pl )H(p 2 ), R t _, < n*:?(l - Pi)H{p t ^) and i? t < Utli 1 ~ Pi)- 

1.1 Our results 

We provide an encoding scheme that for any e > and (Ri, . . . , R t ) in the capacity region 
achieves rate R\ + . . . + R t — e with polynomial time encoding and decoding schemes. 

Theorem 1.1. For any e, 5 > and (Ri, . . . , R t ) G C t there exists an integer N = N(e, 5, t) 
and an explicit construction of a t-write binary WOM code of length N of rate at least 
X)i=i Ri ~ £■ Furthermore, the encoding and decoding functions run in time 

In particular, for (R u . . . , R t ) = (#(^), i+iH(\), ^H^), ^H(\), ^) we give 
a construction of a binary WOM code of rate log(t+ 1) — e. If we wish to achieve a polynomial 
time encoding and decoding then our proof gives the bound N(e,8,t) = (^)° ( ^ ) . If we wish 
to have a short block length, e.g. iV = poly(l/e), then our running time deteriorates and 
becomes N°^ e \ 

A completely analogous construction also works for arbitrary alphabets. However, here 
it is more difficult to explicitly characterize the capacity region. Hence, our theorem for this 
case basically says that given realizable parameters we can construct a scheme achieving 
them (up to e loss in rate). We shall use the following notation. Let Si, . . . ,S t be lower- 
triangular column-stochastic matrices of dimension q x q. We shall think of the (j'2, ji) entry 
of Si (when ji, j 2 G {0,1,2, ... ,q — 1}) as describing the fraction of memory cells that had 
value ji before the ith write and then were written the value j 2 in the ith round, where the 
fraction is computed relative to the number of memory cells holding the value j±. Thus, 
these matrices describe the distribution of values that we expect to see after each write and 
also how values change in the process. For example, in Si only the first column is meaningful 
as the memory initially holds the value 0. We say that Si, . . . , S t is in the capacity region of 
t-write WOM codes over {0, . . . , q — 1} if there exists an encoding scheme realizing Si, . . . ,S t 
(i.e. a scheme that abides the constraints described by the SjS). 

Theorem 1.2. For any e, 5 > and (Si, . . . , S t ) in the capacity region of t-write WOM over 
the alphabet {0, . . . , q — 1} there exists an integer N q = N q (e, 5, t) and an explicit construction 
of a t-write q-ary WOM code of length N q that has rate which is e-close to the rate of the 
scheme realizing Si, . . . ,S t . Furthermore, the encoding and decoding functions run in time 
N 1+s . 

As before we have the bound N g (e,8,t) = (^) 0< -^\ where now the big Oh may depend 
on q as well. Since our proof for the binary setting contains all the relevant ideas, and the 
extension to other alphabets is (more or less) obvious, we shall omit most of the details of 
the non-binary case. 
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1.2 Comparison to the brute force scheme 

The first observation that one makes is that the problem of approaching capacity is, in some 
sense, trivial. This basically follows from the fact that concatenating WOM codes (in the 
sense of string concatenation) does not hurt any of their properties. Thus, if we can find, even 
in a brute force manner, a code of length m that is e-close to capacity, in time T(m), then 
concatenating iV = T(m) copies of this code, gives a code of length Nm whose encoding 
algorithm takes NT(m) = N 2 time. Notice however, that for the brute force algorithm, 
Tim) ~ 2 2 " 1 and so, to get e-close to capacity we need m ~ 1/e and thus N w 2 2 * \ 

In fact, following Rivest and Shamir, Heegard actually showed that a randomized encod- 
ing scheme can achieve capacity for all t |Hee85] . but, naturally, such encoding schemes do 
not have efficient decoding algorithms. 

In view of that, our construction can be seen as giving a big improvement over the brute 
force construction. Indeed, we only require N (t/e) <//e and we give encoding and decoding 
schemes that can be implemented efficiently (as functions of the block length). 

We later discuss a connection with invertible extractors for hamming balls and show that 
an efficient construction of such objects could lead to capacity-achieving WOM codes of 
reasonable block length. 



1.3 Comparison to earlier works 



As we explain in Section 2.1 our work is close in spirit to |Shpl2| , but whereas the latter 
dealt mainly with the case of 2-write WOM codes, here we develop a scheme for any number 
of writes. 

Prior to our work the best deterministic encoding schemes for several writes where ob- 
tained in the works |KYS + 10l IYS12] . As we achieve capacity here and previous constructions 
where at some distance from capacity, which grew with the number of writes, we do not give a 
detailed comparison of the different results and just mentioned that except the work |Shpl2] 
no other work was guaranteed to achieve capacity. (Although we soon discuss a different 
work that docs achieve capacity, but has some failure probability) On the other hand, while 
[KYS + 10| IYS12j did not achieve capacity they did give some multiple write codes of a signif- 
icantly shorter block length than what we achieve here. So, in that respect, they give codes 
that may be more practical than our new construction. 

In |BS12j Burshtein and Strugatski gave a construction of capacity achieving binary 
WOM codes based on polar codes. Their construction is based on the work of [K UlOj that 
uses polar codes for source coding. A small downside of their construction is that it is 
guaranteed to work with high probability, but there is always some small chance of failure. 
In particular, in contrast to our construction, it is not always guaranteed to succeed. On the 
other hand, since |BS12j rely on polar codes that have very efficient encoding and decoding 
schemes, it may be the case that they can make their construction work with a block length 
that is polynomial in 1/e, which is much better than our block length which is exponentially 
large in 1/e. We note however, that it is not clear that their block length is indeed only 
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polynomial in 1/e, as it is a bit difficult to analyze the dependence between distortion- rate 
(which affects the gap to capacity in the ensuing construction of |BS12j ) and block- length 
in the construction of |KU10] . It may very well be the case that although polar codes 
are so efficient, we still encounter a significant blow up in block- length when pushing both 
distortion-rate and failure probability parameters down. Nevertheless, as we said, it may 
still be the case that they can get codes of poly (1/e, log(l/<5)) block-length which are e-close 
to capacity and have failure probability at most 5, in which case these codes will have more 
reasonable parameters than what we achieve here. 

1.4 Notation 

For a k x m matrix A and a subset S C [m] we let A\s be the k x \S\ sub-matrix of A 
that contains only the columns that appear in S. For a length m vector y and a subset 
S C [m] we denote with y\s the vector that has exactly \S\ coordinates, which is obtained 
by "throwing away" all coordinates of y outside S. 

We denote [k] = {1, . . . , k}. for a binary vector w G B n we denote by weight (w), the 
weight of w, the number of nonzero entries of w. 



2 Capacity achieving t-write WOM codes 
2.1 Our approach 



We describe our technique for proving Theorem 1.1 We start by explaining the connection 
to the 2- write WOM codes of |Shpl2| , which themselves follow ideas from |WulOt |YKS + 10] . 
Similarly to [RS82] . in the first round jWulOl lYKS+lOi |Shpl2| think of a message as 



a subset S C [n] of size pn and encode it by its characteristic vector. This is also what 
we do in this paper. Clearly in this step we can transmit H(p)n bits of information. (I.e. 
log « H(p)n.) 

For the second round assume that we already sent a message S C [n]. I.e. we have 
already wrote pn locations. Note that in order to match the capacity we should find a 
way to optimally use the remaining (1 — p)n locations in order to transmit (1 — p — o(l))n 
many bits. |Shpl2| handled this by giving a collection of binary codes that are, in some 
sense, MDS codes on average. Namely, a collection of (less than) 2 n matrices {A} of size 
(1 — p — e)n x n such that for any subset S C [n], of size pn, all but a fraction 2~ en of the 
matrices Ai, satisfy that -Aj|r n ]\s has full row rank (i.e. rank (1 — p — e)n). Now, if in the 
first round we transmitted a word w corresponding to a subset S C [n] of size pn then in the 
second round we find a matrix Ai such that Aj|r n i\g has full row rank. Now, given a word 
x of length (1 — p)n to write in the second round, we find (by solving a system of linear 
equations) a vector y > w such that Aiy = x. We then write y to memory. Note, however, 
that we also have to write the index i to memory so the decoder knows how to decode y (the 
decoding function simply multiplies y by Ai). However, since i ranges over a set of size 2 n , 
its index has length n so writing it to memory reduces the rate significantly. Furthermore, to 
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find the good Ai we need to range over an exponentially large space, namely, a space of size 
2 n . Nevertheless, |Shpl2| showed that we can use the same Ai for encoding many messages 
Xi in parallel, so by concatenating N = exp(n) many copies of this basic construction, we 
obtain a code with the required properties. Indeed, now, a running time of exp(n) is merely 
polynomial in N. 

The approach that we apply here is similar but with a few notable changes. Since we 
wish to have more than two write rounds we cannot use all the (1 — p)n bits left after 
writing the subset S to memory. Instead we have to leave many unwritten bits for the third 
round and then to the fourth round etc. To do so, and still follow the previous approach, 
we would like to find a sparse vector y > w that still satisfies Aiy = x, for some Ai in 
the collection. For certain, to ensure that we can find such a y we must know that such y 
indeed exists. Namely, that there is some Aj that has such a sparse solution above w to the 
equality Aiy = x. Moreover, if we wish to continue with the analogy then we need to find 
such Ai that will be good simultaneously to many equations of the form Aiyj = Xj where 
Uj > Wj. Indeed this is what we do. While the work |Sh pl2| used the Wozencraft ensemble 
of matrices, in this work we use a universal family of pairwise independent hash-functions. 



(We describe the family and its properties in Section 2.2 ) We show that in this family we 
are always guaranteed to find a matrix for which there exist sparse solutions (that sit above 
the relevant lOj's) to several such equations. This basically follows from the fact that this 
family is an extractor with an exponentially small error. (We state the required properties, 



which are weaker than what can actually be proved, in Lemma 2.1.) However, it is still not 
clear how to find the solutions yj even if they exist. The point is that by solving a system of 
linear equations we are not guaranteed to find a sparse solution. Here we again use the fact 
that we are allowed to concatenate many copies of our construction in parallel. Indeed, as 
finding a good Ai already requires exponential time, we can let ourselves look for a solution 
yj in exponential time without hurting the already exponential running time. At the end, 
when we concatenate exponentially many copies of this basic construction, it will look as if 
we are solving in a brute force manner a problem on a logarithmically small space, which 
at the end is not too terrible. The effect of concatenating exponentially many copies of the 
basic construction, besides making the running time a small polynomial in the block length, 
is that the output block length is exponential in 1/e, where e is the gap to capacity that we 
achieve. 

We note that the construction of |Shpl2] suffered a similar blow-up in block length, from 
essentially the same reasons. The same is also true for the best codes of |YS12j as their 
construction relies on some of the tools developed in |Shpl2| . 

2.2 Hash functions 

Our construction will make use of the following family of pairwise independent hash func- 
tions. It will be instructive to think of the field F 2 ™ both as a field and as a vector space 
over F2. Thus, each vector x G F 2 « can be naturally described as a length n vector, after 
we fix some basis to F 2 «/F 2 . For a,b G F 2 n we define the map H™f ,e : {0, l} n — >■ {0, l} k ~ e 
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as H2f ,e {x) = (ax + b)[k-e\- In words, we compute the affine transformation ax + b in ¥ 2 «, 
represent it as an n-bit vector using the natural map and then keep the first k — I bits of 
this vector. Note that we think of x both as an n-bit vector and as an element of ¥ 2 ™ . We 
represent this family by TL n ' k,£ . Namely 

n n ' k ' i =[^' e \a,be¥ 2n y 

We will use the following special case of the leftover hash lemma of [ILL89J. 

Lemma 2.1 (Leftover hash lemma }ILL89j ). Let k,£,n be integers. Let Y C {0, 1}™ be a set 

of size 2 k . Then, the distribution (a,b, H2' k ' e {y)), obtained by picking a, b G F 2 ™ and y G F 
independently and uniformly at random, is 2~ i l 2 close, in statistical distance^ to the uniform 
distribution on {0, \} 2n+k ~ l . 

Corollary 2.2. Let Y C {0, 1}™ be a set of size 2 k . The fraction of H e n n ' k/ such that 
\H(Y)\ < 2 k - £ (l - 2- £ / 4 ) is at most 2~ £ / 4 . 



Proof. Observe that if |-ff(y)| < 2 k ~ e (l — 2~ £ / 4 ) then the statistical distance between H(Y) 
and the uniform distribution on {0, l} k ~ e is at least 2~ £ / 4 . If there is a fraction > 2~ e ^ 4 of 
such "bad" H then the statistical distance between the uniform distributions on ( H, H (Y)) 
and J{ n ' k ' e x {0, l} k ~ e is larger than 2~ e ^ 4 ■ 2~ £ / 4 = 2~ e ^ 2 , in contradiction to Lemma 

The following corollary will be very useful for us. 



2.1 



□ 



Corollary 2.3. Let £, k, n, m be integers such that t < k < n and m < 2 e ^. Let Yi, . . . , Y m C 
{0, l} n be sets of size \Yi\, . . . , \Y m \ > 2 k . Then, for any x%, . . . , x m G {0, l} k ~ e there exists 
H G W 1 ^^ and {yi G Y^} such that for all i, H(yi) = Xi. 

In words, the corollary says that if m < 2 e ^ then we can find an H G T-l n ' k ' e such that 
for all i, Xi is in the image of H when applied to Yi (i.e. X{ G H(Yi)). 



Proof. By Corollary 2.2 and the union bound we get that there is some H™' k,e G "H™' fc ' £ such 



that for all i, 1^ (^)1 > ^i 1 ~ 2 ^ /4 )- Denote X t = H^f 'iYi). Observe, that by the 
union bound, there exists v G {0, l) k ~ l such that for all i, x^ © v G Thus, there exist 
{yi G Yi} such that for all i, H™' k,e (yi) = Xi@ v. In other words, for all i, ^mvoo^^ = Xi " 

where by v o we mean a length n vector that is equal to v on the first k — i bits and is zero 
elsewhere. H = H n ' k ' e -, is a function satisfying the claim. □ 



The statistical distance between two distribution D\ and D2 defined over a set X is max^cx | P?D! [T] — 
Pr£> 2 [T] j . Note that this is half the £1 distance between D\ and D2 ■ 
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2.3 The basic construction 



We now describe a basic construction that will be the building block of our final encoding 
scheme. The final construction will be composed of concatenating several blocks of the basic 
construction together. To make reading easier, we drop all notation of floors and ceilings, 
but the reader should bear in mind that not all numbers that we may encounter in the 
analysis are integers and so some rounding takes place, but we choose parameters in such a 
way that we absorb all loss created by such rounding. 

Lemma 2.4. For any e > 0, (R±, . . . , R t ) G C t and 1 < t < t the following holds. Let n 
be an integer such that n = clog(l/e)/e 7 for some constant c > 20. Let k,£,m be integers 
satisfying k = (R to - e/3)n, £ = en/3 = clog(l/e)/3 and m < 2^ 4 = (l/e) c / 12 . Let 
Wi, . . . ,w m G {0, l} n be such that for all i: if to — 1 then weight{uii) = 0, otherwise, 

weight{wi) < (\ — Y^j=i{^- ~ Pj) ) n - Let x±, . . . ,x m G {0, l} k ~ e . Then, there exists H ta G 

'H n ' k ' e and yi, . . . , y m G {0, 1}™ with the following properties: 

1. For all i G [m], yi > Wi. 



3. For all i G [m], H k) {yi) = x { . 

Furthermore, we can find such H to and y±, . . . , y m in time 2 4n . 

Intuitively, one should think of each Wi in the statement of the lemma as the state of the 
memory after to — 1 rounds of writing (i.e. before round to)- The map H to that the lemma 
guarantees will be the encoding map used in round to and each yi will be the word replacing 
Wi in memory. (We are working in parallel with several Wi and yi but with a single H to .) 



2. For all i G [m], weight^) < 1 1 - Ujlii 1 ~ Pj)) n - 



Proof. Let 



Yi = ly G {0, l} n \y>Wi and weight^) < 1 - ][[(! - Pj ) 





As weight (wj) < ( 1 — YljLi — Pj)) n we have that 



YA > 



( 



If?=i (1- Pi) -Pto-n. 



n — weight (wj) 



) 



> 



( 



n-r^i-P.o-Pto^. 



which by Stirling's formula can be lower bounded by 
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We can now apply Corollary 2.3 to find H to G % n > fc > £ an d {y^ G 5^} such that for all i, 
Ht {yi) = Xi. As yi G we see that all the three requirements in the lemma hold. 

As for the furthermore part, we note that since we are guaranteed that such H and {yi} 
exist, we can find them in a brute-force manner in time \H n ' k ' \ ■ Y^tLi 1^1 — m2 3n < 2 4n . □ 

Note that we need n to satisfy en > 31og(n) + 9. Thus, n > 201og(l/e)/e suffices. 

The lemma basically tells us that if we have m memory blocks, each of length n, then no 
matter what we wrote to each memory block in the first to — 1 rounds, we can find suitable 
{yi} for writing in the next round. Notice that in order to decode Xi one simply needs to 
compute H to (yi). However, we somehow need to communicate H to to the decoder as it may 
depend on the content of the memory after the first t^ — l rounds. In order to do so we write 
the 2n-bit index of H to on fresh memory cells. (Recall that each H G 'H n > k ' e is defined by a 
pair (a, b) G F 2 « x F 2 ™.) Wrapping everything up we now define the basic building block of 
our t-write WOM scheme. 

Basic WOM scheme: For any given e > and {R\, . . . , Rt) G C* we define the following 
encoding scheme. Let c > 20 be an integer satisfying 6t 2 < (t/e) c / 12-1 . Let n = c£log(t/e)/e 
be an integer. Let k 2 , . . . , k t , £, m be integers satisfying ki = (Ri — e/3t)n, £ = en/3t = 
clog(t/e)/3 and m = 2 £ ^ A — 1 = [t/e) c ^ 12 — 1. Let iVo = 2n(t — 1) +mn. Our encoding scheme 
will write to memory of length iVo- We shall think of the Nq memory bits as being composed 
of m blocks of n bits where we write the "actual" words and additional t — 1 blocks of 2n 
bits each where we store information that will help us decode the stored words. 

• First round: A message in the first round is given as a list of m subsets of [n], each of 
size at most p\n. We store the zth subset to the ith n-bit block by simply writing 1 on 
all indices belonging to the set and elsewhere. It is clear how to decode the message 
written in the first round. 

• Round j: The input message for round j is composed of m words xi,...,x m G 
{0, Assume that the m n-bit memory blocks contain the words w±, . . . , w m . We 



first find Hj and yi, . . . , y m as guaranteed by Lemma 2.4 We write to the zth n-bit 
memory block and we also write the index of Hj to the j — 1 block of length 2n. To 
decode we simply read the index of Hj from the j — 1th 2n-bit block, and apply Hj on 
each yi to get Xj. 



Analysis: 



Correctness: By induction one can easily show that if w was written to an n-bit 
memory block at round to > 1 then weight(w) < ^1 — n$°=i(l — Pj)J n - Thus, at the 



jth round the conditions of Lemma 2.4 are satisfied (assuming that we pick n large 



enough) and therefore we can find Hj and {yi} as required. 
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Rate: We first note that by our choice of parameters 

mn mn 2(t — 1) 2i i/o 

J% = (mn + 2n(t - 1)) = ~ m + 2(t - 1) > ~ (i/e) c / 12 > ~ C/ 

In the first round we write a word of weight pin. Thus the rate of this round is 

mlo &{(*n)) (H( Pl )n-\og(n))m ^ (Hfa) - nm e e e 
W > No > Wo " {Rl ~^ ){1 ~3t ) > Rl ~~f 

In the jth round we encode m messages each of length kj — £. Thus, the total rate is 



m ( k j ~ I) ^ m ((Rj ~ e/3t) - e/3t)n . 2e s 



e , „ e 



> '- 1 '- — '— >(R i )(1 )>Ri- ■ 

N N V 3 3i A 3t J 3 t 

It follows that the total rate is at least R± + . . . + Rt — if = -Ri + . . . + Rt — e. 



• Encoding Complexity: by Lemma 2A each round can be completed in time 2 4n = 

(i/e) 4c ^. 

• Decoding complexity: To decode a message written in the jth round (assume j > 1 
as the case j = 1 is clear) we simply need to perform a product of an (kj — £) x n 
matrix with m n-bit vectors (we can think of each H a b as an affine transformation and 
associate to it a (k — I) x n matrix and a (kj — £) shift vector that completely define 
it). Thus, the decoding running time is poly(kmn) = (t/t)°^^\f\ 

We summarize our conclusions in the following theorem. 

Theorem 2.1. For any given e > and (Ri, . . . , R t ) G C( let c > 20 an integer satisfying 
6i 2 < (t/eY 112 - 1 . Let n = cilog(i/e)/e and m = (i/e) c/12 - 1 be integers. Let N = 
2n(t — 1) +?rm. Then, the scheme described above is a t-write WOM scheme of block length 
No, with rate larger than R\ + . . . + R t — t, encoding complexity of (i/e) 4c '/ e and decoding 
complexity (i/e) 0(c ). 

2.4 The full construction 

The full construction is obtained by concatenating (in the sense of string concatenation) 
several blocks of the basic construction together. 

Theorem 2.2. Given a, e > and (R\, . . . , Rt) G C t let c > 20 an integer satisfying 6i 2 < 
(i/e) c,/12_1 . Let n = ct\og(t/e)/e, m = (i/e) c / 12 — 1 and N = 2n(t — 1) + mn be integers. Set 
Ni = 2 a- 1 ■ Nq. Then, there is an explicit t-write WOM scheme of block length iV 1; with rate 
larger than Ri + . . . + R t — e, encoding complexity of N® and decoding complexity jV 1 1+ °^ a<E// ^ . 

Here we assume that the field F2" is explicitly given by an irreducible polynomial, so given a € ¥2" we 
can find the n x n matrix A over F 2 satisfying Ax = ax for all x G F 2 n (when we abuse notation and think 
of x both as an n-bit vector and field element) in time poly(n). 
6 We do not try to optimize running time here. 
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Proof. Let n% = (Ni/Nq) = 2 4n /( a ~ 1 ). Our scheme is composed of concatenating n\ copies of 
the basic construction. Thus, the block length that we obtain is n\ ■ No — N\. It is clear that 
concatenation does not change the rate and the effect on the complexity is that we need to 
repeat all operations rt\ times. The claimed running times now follow from Theorem 2A_ □ 

1 + 5. Note that 



Theorem 1.1 now follows by appropriately choosing a 



Ni = rii ■ N = 0(rii ■ n ■ m) 




c/12 



ct \og{t/t 



03 



2.5 Larger alphabets 



We only give a sketch of the proof of Theorem L2 as it is nearly identical to the proof of 
Theorem 11.11 



As before we shall think of the field ¥ q n both as a field and as an n-dimensional vector 
space over ¥ q . For a, b <G ¥ q n we define the map H™f' e : ¥ q — > ¥ q ~ e as H™f' e (x) = {ax+b)[k~i\- 



We also set 



H 



n,k,i 



H n a f\a,be¥ q 



The first step is to note the following analogs to Lemma 2.1 and Corollary 2.3 



Lemma 2.5 (Leftover hash lemma over ¥ q |ILL89j ). Let k,£,n be integers. Let Y C F" be 
a set of size q k . Then, the distribution (a,b,H™f ,e (y)), obtained by picking a,b 6 F g n and 
y E Y independently and uniformly at random, is q^ e ^ 2 close, in statistical distance, to the 
uniform distribution on Y"j, n+k ~ e . 



Corollary 2.6. Let q 

Yi,...,Y r 



fc, n, m be integers such that i < k < n and m < q e 



C F™ be sets of size \Yi\ 



\Y m \ > q . Then, for any xi, 



Xr 



exists H G T-L q ,k,e and {yi 6 Yi} such that for all i, H{yi 



Let 
there 



X i 



Given Corollary 2.6, we can construct, in a similar manner to Sections 2.3 and 2.4 
schemes for t-write g-ary WOM codes. The only difference is that the matrix Si (recall 



discussion before the statement of Theorem 1.2) will be used to define the sources Y in 



which we will be looking for solutions in the ith write. Namely, give a word w in memory, 
the relevant codewords for the ith write will be those that have a fraction (Si)^^ coordinates 
labeled ji among all coordinates where w had value ji, namely, we shall work with the sources 



Y t {w) = {w' | #{6 



j 2 and w b = ji} w (Si) j2:jl ■ #{6 | w b = . 



3 Discussion 

In this work we showed how to construct capacity achieving t-write WOM codes. The main 
drawback of our codes is that the block length is exponential in 1/e, where e is the gap to 
capacity. It is an interesting question, that may have practical significance, to construct 
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capacity achieving codes with a much shorter block length iV = poly(l/e). If we would 
like to get such a result using similar techniques, then, in particular, we will need a way to 
efficiently find sparse solutions to systems of linear equations of restricted forms. We know 
that finding sparse solutions to general linear systems is NP-hard, but as the matrices that 
we used for our construction are very structured it does not sound improbable that such an 
algorithm should exist. Moreover, one can also imagine a different set of matrices that can 
be used for decoding for which sparse solutions can be found efficiently. 

Another way to think of our use of the universal hash family is that what we actually 
used is a very good extractor (for background on extractors we refer the reader to |Sha02j). 
Namely, an extractor with an exponentially small error. We note that we actually do not 
need such good extractors. We merely gave this construction as it is easier to explain. In 
particular, one can use |Rao09l ISha08] to replace the ensemble of matrices with a single 
function (i.e. with a seedless extractor for several sources). While this will save a bit on 
the block length (though will not affect it in a significant manner) the main hurdle which is 
efficiently inverting the extractor still remains. Thus, it seems that the main challenge, as 
far as our approach goes, is to efficiently construct seedless extractors (or extractors with an 
exponentially small error) that can be efficiently inverted, assuming that the source has some 
known "nice" structure. In our case all the sources that we deal with are highly structured 
- they contain all vectors of a certain weight that "sit above" a given vector. In a sense, this 
is the "positive" part of the hamming ball centered at the given vector. We think that this 
is an interesting challenge and we expect that developments regarding this question will lead 
to improved constructions of WOM codes. 

|Shpl2| also considered the related problem of designing encoding schemes for defective 
memory and gave a scheme based on the extractor of Raz, Reingold and Vadhan [RRV02J . 
The disadvantage is that for the scheme to work |Shpl2| needed to assume that there are 
poly(logn) "clean bits" that are known to both the encoder and decoder. Recently Gabizon 
and Shaltiel considered the problem and showed how to obtain capacity achieving encoding 
schemes from zero-error seedless dispersers for affine sources [GSI2]Q Similarly to our work, 
[GS12j needed their disperser to be efficiently invertible. Since they were dealing with affine 
sources (and not with Hamming balls as we do here) they managed to achieve this by cleverly 
combining a search over a small space with solving a system of linear equations. We also note 
that the encoding function of [G ST2] is randomized (with expected running time poly(ra)), 
whereas the scheme of |Shpl2| , and the one that we give in this paper, are deterministic. 
Nevertheless, it may be possible that using techniques similar to those of |GS12j one may be 
able to improve on the block-length of our construction. 

7 Roughly, a zero-error disperser for affine sources is a function /{0, 1}™ — > {0, f } m such that any affine 
space V, of high enough dimension, is mapped onto {0, f} m . That is, for any such V, f(V) = {0, f } m . 
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